The paper was to reduce biased estimation using new approach (Penalized Maximum likelihood Estimation (PMLE) method) in Logistic Regression. For this aim, unreal four small data sets were randomly generated. Maximum likelihood Estimation (MLE) and PMLE methods were applied and compared for separation case including biased estimation in Logistic Regression when one of the cells in 2 x 2 tables becomes equal to zero (separation problem). Parameters1 and their standard error obtained by using MLE for four data sets were 12.56±257.8, 13.46±264.3, 13.42±210.3, and 13.41±180.4, respectively, meaning that MLE’s are biased estimates. Corresponding values for PMLE method were found 2.28 ± 1.81, 3.05 ± 1.59, 3.45 ± 1.53, and 3.45 ± 1.53, respectively, meaning that PMLE’s was unbiased estimates. It is clear that standard error value for data set 1 reduced from 257.8 to 1.81 when using PMLE method for separation problem. According to PMLE method, the odds of being coronary heart disease risk for smokers were increased 21.08 times than that for non-smokers smoking in data set 2, which is significant at 1% level. The odds of being coronary heart disease risk for smokers were increased 31.63 times than that for non-smokers in data set 3 (P < 0.001). The odds of being coronary heart disease risk for smokers were increased 41.93 times than that for non-smokers in data set 4. When one of the cells in 2 x 2 contingency tables becomes equal to zero, PMLE was more superior to MLE method because PMLE method may be performed unbiased (reliable) estimation.

Newton's method, which is also known as the Newton Raphson algorithm, as one of the most efficient numerical methods in mathematics, are known as a method and approach for the root approximation of nonlinear equations. After reviewing and interpretation of this method, one of the most widely used in the statistics, i. e. estimation of the unknown parameters by Maximum likelihood method is presented in this paper. To facilitate the transfer of concepts, the article includes several different numerical examples and computer programs.

The present work focuses on the second order Markov chain model which arises in a variety of settings and is well-suited to be modeled in many applications. The efficiency of the Maximum quasi-likelihood estimators with the full Maximum likelihood estimators for second order Markov chain models are given, besides the limiting normality results on the asymptotic properties of the associated estimates. Some efficiency calculations are also given to discuss the feasibility and computational complexity of the QL approach relative to the full likelihood approach.